Global Optimization Methods for Extended Fisher Discriminant Analysis
نویسندگان
چکیده
The Fisher discriminant analysis (FDA) is a common technique for binary classification. A parametrized extension, which we call the extended FDA, has been introduced from the viewpoint of robust optimization. In this work, we first give a new probabilistic interpretation of the extended FDA. We then develop algorithms for solving an optimization problem that arises from the extended FDA: computing the distance between a point and the surface of an ellipsoid. We solve this problem via the KKT points, which we show are obtained by solving a generalized eigenvalue problem. We speed up the algorithm by taking advantage of the matrix structure and proving that a globally optimal solution is a KKT point with the smallest Lagrange multiplier, which can be computed efficiently as the leftmost eigenvalue. Numerical experiments illustrate the efficiency and effectiveness of the extended FDA model combined with our algorithm.
منابع مشابه
Robust Fisher Discriminant Analysis
Fisher linear discriminant analysis (LDA) can be sensitive to the problem data. Robust Fisher LDA can systematically alleviate the sensitivity problem by explicitly incorporating a model of data uncertainty in a classification problem and optimizing for the worst-case scenario under this model. The main contribution of this paper is show that with general convex uncertainty models on the proble...
متن کاملMultiview Fisher Discriminant Analysis
CCA can be seen as a multiview extension of PCA, in which information from two sources is used for learning by finding a subspace in which the two views are most correlated. However PCA, and by extension CCA, does not use label information. Fisher Discriminant Analysis uses label information to find informative projections, which can be more informative in supervised learning settings. We show ...
متن کاملFisher Discriminant Analysis (FDA), a supervised feature reduction method in seismic object detection
Automatic processes on seismic data using pattern recognition is one of the interesting fields in geophysical data interpretation. One part is the seismic object detection using different supervised classification methods that finally has an output as a probability cube. Object detection process starts with generating a pickset of two classes labeled as object and non-object and then selecting ...
متن کاملLinear Discriminant Dimensionality Reduction
Fisher criterion has achieved great success in dimensionality reduction. Two representative methods based on Fisher criterion are Fisher Score and Linear Discriminant Analysis (LDA). The former is developed for feature selection while the latter is designed for subspace learning. In the past decade, these two approaches are often studied independently. In this paper, based on the observation th...
متن کاملOn a self-consistent-field-like iteration for maximizing the sum of the Rayleigh quotients
In this paper, we consider efficient methods for maximizing x >Bx x>Wx + x>Dx over the unit sphere, where B,D are symmetric matrices, and W is symmetric and positive definite. This problem can arise in the downlink of a multi-user MIMO system and in the sparse Fisher discriminant analysis in pattern recognition. It is already known that the problem of finding a global maximizer is closely assoc...
متن کامل